[python] Create xarray backend for DataArray types
#3243
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3243 +/- ##
==========================================
+ Coverage 83.81% 84.92% +1.11%
==========================================
Files 51 52 +1
Lines 5578 5613 +35
==========================================
+ Hits 4675 4767 +92
+ Misses 903 846 -57
Flags with carried forward coverage won't be shown. Click here to find out more.
|
johnkerl
changed the title
jp-dark
force-pushed
the
dark/xarray-backend/sc-57982
branch
from
9bee8a8 to
bf94672
Compare
johnkerl
changed the title
xarray backend for DataArray types
There was a problem hiding this comment.
I would like to see some demonstration or ideally testing that makes sure that memory usage matches our expectations with lazy loading.
As ideas for testing, I can recommend memray's pytest plugin or even just logging whenever a tile is accessed and making sure you aren't accessing unneeded tiles when using the xarray interface.
jp-dark
force-pushed
the
dark/xarray-backend/sc-57982
branch
from
e9bcc79 to
ef6ac6f
Compare
|
^ I would be happy to have performance monitoring pushed to a later PR if it gets us to |
I added issue #3267 to track this. |
Add an xarray DataStore implementation for opening a SOMA DenseNDArray as a xarray Dataset with a single DataArray.
Co-authored-by: Isaac Virshup <[email protected]>
jp-dark
force-pushed
the
dark/xarray-backend/sc-57982
branch
from
4df6528 to
e7ddebc
Compare
There was a problem hiding this comment.
LGTM with suggested changes. (I could 'request changes' but I trust you to make them -- if you accept my suggestions as-is. If not, happy to iterate in further conversation.)
My main point of confusion reading the unit-test cases was keeping track what is the datatype of each variable. My comments are all around naming that stands out more clearly.
Co-authored-by: John Kerl <[email protected]>
Issue: Closes #3242
Changes:
Adds an Xarray
BackendArraywrapper andDataStorefor theSOMADenseNDArray. This allows the user to open a singleSOMADenseNDArrayas an XarrayDatasetwith a singleDataArray.This is important because we already use (and need) the SpatialData package, and it uses Xarray objects.